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Our mathematics cognitive technology noticing framework is based on professional noticing and 
curricular noticing frameworks and data collected in a study that explored how secondary 
mathematics teachers evaluate technology. Our participants displayed three categories of noticing: 
attention to features of technology, interpretation of the features, and response to these features. We 
believe this framework can potentially allow researchers to document the evolution of teachers’ 
evaluation of mathematical cognitive technologies and examine how the inclusion or exclusion of 
particular features of technology influences their evaluation and subsequent selection of technology. 
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As technology is an essential tool for learning mathematics in the 21st century (NCTM, 2014) 
and is becoming more ubiquitous in the classroom, mathematics teachers are being asked to evaluate, 
select and implement quality technological tools to use with their students. However, little is known 
about how teachers select technology, particularly mathematical cognitive technologies (MCTs), 
digital technological tools that can be used to teach and learn mathematical concepts “that [help] 
transcend the limitations of the mind (e.g., attention to goals, short-term memory span) in thinking, 
learning, and problem-solving activities” (Pea, 1987, p. 91). The purpose of this paper is to propose a 
framework for thinking about what teachers notice when they evaluate, select, and modify 
technology to use in their mathematics classrooms. 


Background 

As an everyday term, noticing 1s used to refer to particular observations that one makes (Sherin, 
Jacobs, & Philipp, 2011). In the field of mathematics education, researchers specifically use the 
phrase teacher noticing to refer to what teachers attend to in a classroom situation and how they 
make sense of what is observed using their knowledge of mathematics teaching and learning and the 
context being observed (Blomberg, Stiirmer, & Seidel, 2011; Dreher & Kuntze, 2015; van Es & 
Sherin, 2002). Jacobs, Lamb, and Philipp (2010) and McDuffie et al. (2013) extended previous views 
of noticing to include instructional responses teachers make. Jacobs et al. conceptualized 
professional noticing of children’s mathematical thinking as three interrelated and cyclic 
components: attending to students’ thinking, interpreting their thinking, and deciding how to respond 
based on their understanding. Researchers in mathematics education extended Jacobs et al.’s noticing 
framework to connect to equity (Wager, 2014) and curricular materials (Males, Earnest, Dietiker, & 
Amador, 2015). 

By drawing upon constructs of the study of Jacobs et al. (2010) for professional noticing of 
children’s mathematical thinking, Males et al. (2015) developed the curricular noticing framework to 
provide support for examining what teachers notice in curriculum materials, how they make sense of 
what they attend to, and what actions they make based on their observation of curriculum materials. 
Providing findings of four exploratory studies by prospective elementary and secondary teachers, 
Males and colleagues demonstrated that mathematics method courses could support prospective 
teachers in developing noticing skills of curriculum materials. They claimed that the curricular 
noticing framework might provide a lens for describing the mechanisms for teacher decision-making 
with curricular materials. 

Once considered an add-on to the curriculum, technology has a strong presence in many of 
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today’s mathematics classrooms and teachers are being asked to regularly evaluate technology and 
integrate it into the curriculum. Previous research on teachers’ evaluation of technology to teach 
mathematics has focused on the criteria created and analysis performed by prospective elementary 
teachers (e.g., Battey, Kafai, & Franke, 2005). Battey et al. (2005) studied prospective elementary 
teachers’ criteria for evaluating rational number software. The researchers found the prospective 
teachers’ criteria focused more on surface features of the software (e.g., clarity of directions, clear 
visual presentation, and ease of use) than on specific mathematics content or how technology could 
support students’ learning. Johnston and Suh (2009) studied prospective elementary teachers’ 
planning for mathematics instruction with technology. They found that only a few of their 
participants selected MCTs to use in their lesson plans and they considered technology to be a 
beneficial tool for developing students’ conceptual learning or providing a visual representation. The 
prospective elementary teachers who selected review games or non-MCTs (e.g., digital cameras or 
Smart Boards) indicated that a benefit of using technology is that it is a fun, engaging, or motivating 
tool. The studies by Johnston and Suh and Battey et al. indicated that, in general, prospective 
elementary teachers seem to select and use technology based on student engagement, surface features 
of the software, and motivation rather than developing students’ understanding of mathematical 
content. Although these studies provide evidence of the criteria prospective elementary teaches use to 
select technology and what the teachers believe to be the main benefits of using technology, the 
authors of the studies did not provide much insight into how the teachers actually evaluated these 
technologies to teach mathematics. In particular, what features did the teachers attend to? How did 
they interpret these features? What were their responses? We believe our framework builds upon and 
extends the actions that other researchers have identified in a way that specifically characterizes what 
teachers notice when evaluating MCTs as an important subset of their curricular noticing. 


The Mathematical Cognitive Technology Noticing Framework 

The framework is based heavily on the work of Jacobs et al. (2010) in professional noticing and 
the work of Males et al. (2015) in curricular noticing. In both works, the authors focus on three 
categories of teachers’ action: Attending, Interpreting, and Responding. If we can agree the 
evaluating technology is a subset of curricular noticing, then it seems natural to use these same 
categories. However, the codes would be slightly different from those proposed by Males et al. 
because we focus specifically on the evaluation of MCTs while they provide a more general 
framework on curricular noticing. To develop the codes, we observed and analyzed four trios of 
teachers as they evaluated and compared four MCTs to use to teach the triangle inequality theorem to 
eighth grade students. Each of the four MCTs was an online pre-constructed dynamic geometry 
sketch. In the transcript, we noted when the teachers seemed to be attending, interpreting, and 
responding and created codes based on their words and actions within each of these categories. As 
we continued our analysis, we modified and collapsed codes in order to have meaningful codes and 
categories. The three categories of noticing, along with the codes of each, constitute the mathematical 
cognitive technology noticing framework (see Table 1). 
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Table 1: Mathematical Cognitive Technology Noticing Framework 
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instruction 
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: . feature of the MCT : . 
and instructions Value ‘ . Adapt adjust the given 
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the screen : . 
learning without 
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decades Attends to the Diferentiation all students would 


layout of the MCT 


Mathematics 


be able to use the 
MCT productively 


Interprets the 
display and ways 
students interact 

with mathematical 
objects 


As an example, consider the following analysis performed by Mary, Abby, and Bob (all names 
are pseudonyms) who had earned undergraduate degrees in secondary mathematics education and 
were pursuing Masters degrees in mathematics education. All were currently in their first year of 
teaching mathematics at a middle school or a high school. In this example, Mary, Abby, and Bob 
were evaluating an internet-based pre-constructed sketch (Garrison, n.d.) that students could use to 
develop the triangle inequality theorem (see Figure 1). The website provides brief directions on how 
to use the applet and a question asks users to generalize a theorem based on their experiences using 
the applet. Two sets of objects are displayed at all times: a set of sliders (pink, green, and yellow) and 
a set of segments whose lengths and colors are linked to the corresponding sliders. Users are able to 
change the length of segments by using sliders that range from 0 to 10 with 0.5 increments. The 
applet’s appearance is very different depending on whether a triangle can be formed with a given set 
of segment lengths. When a triangle can be formed, two additional objects appear: a smiley face and 
a tan triangle (see Figure la). The tan triangle shows the user where the pink and green segments 
would need to be dragged in order for the triangle to be created. The user can drag each endpoint of 
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the two segments (pink and green) to make the triangle. The tan triangle also has markings of the 
interior angles of the triangle which are named angles alpha, beta, and gamma. However, the 
measures of these angles are not displayed and the user cannot measure them. If the conditions for a 
triangle are not satisfied, the applet does not make the tan triangle and the smiley face is not 
displayed (see Figure 1b). Users can drag each endpoint of the two segments to check the fact that a 
triangle cannot be formed. The applet’s website does not provide any further assistance or directions 
other than the name of the applet’s creator and a link to the Geogebra website. 


Discovering the Triangle Inequality Theorem Discovering the Triangle Inequality Theorem 
Adjust sliders a, b, and c by dragging with your mouse until a triangle is formed. Adjust sliders a, b, and c by dragging with your mouse until a triangle is formed. 
If no triangle is formed, drag the red points to show that the third vertex of atriangle _If no triangle is formed, drag the red points to show that the third vertex of a triangle 
can not be formed. can not be formed. 
a=4 a=15 
——_—_@—_—_ =O 
b=5.5 b=5.5 
———9--—- ———o——_. 
c=9 c=9 
—————_°- —————o- 
5.5 C id 5.5 
1.5 
A 9 B A 9 B 


What conclusion can you make about the side lengths necessary to form a triangle? ~~ What conclusion can you make about the side lengths necessary to form a triangle? 
Write a theorem. Write a theorem. 


Mrs. Garrison, Created with GeoGebra Mrs. Garrison, Created with GeoGebra 


(a) (b) 


Figure 1: MCT Analyzed by Mary, Abby, and Bob when the length of the segments do (a) and do 
not (b) form a triangle. 


Mary, Bob, and Abby spent over 12 minutes evaluating this applet. Given the space limitations, it 
is impossible for us to provide the transcript in its entirety. Instead, we provide the following 
narrative with direct quotations used as much as possible. Opening the website, Abby read the 
instructions aloud and Bob pointed out that the triangle is present and the angles are marked with 
Greek letters. Bob briefly dragged the sliders and said, “It gives you the awesome smiley face. It 
would be better if it gave you a frowny face if it didn’t work.” As they determined whether the applet 
engages students in meaningful ways, Abby said, “it’s pretty straightforward, except for the smiley 
face and we’ll just drag {it}.” Bob responded, “Well it’s unnecessary that it [instruction] tells you to 
move the line segments, since it already creates it, so it’s kind of unnecessary.” Bob continued to 
explore the applet and supported his previous claims stating, 


Bob: [Dragging segments] See here when they’re the same, it does, it shows you if you move 
your triangle, and then it shows point C as, well, so it shows that. But then there’s still no 
new, I guess, to see that you’d move it here. 

Mary: Yeah. 

Bob: But with the ones that actually make triangles, there’s no need to move anywhere. It’s very 
obvious. 

Mary: It’s very lame. 
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Changing the foci of the conversation to the specific benefits and limitations of the applet, Abby 
said, “Okay, yes the use of three colors distinguishes the different segments. Um, when a triangle is 
formed a smiley face appears encouraging students, encouraging students that they achieved a 
triangle. They create a triangle.” Mary stated a benefit is that 1t provides instructions on how to use 
the applet. Abby considered whether the angle markings are beneficial. She said, “In terms of the 
mathematics. I think that in this case with the angle measures, since we’re not really concerned with 
angle measures, it could potentially distract [from the triangle inequality theorem].” She continued, 
“They [Angle markings] are not, they are not essential for the development of this theorem.” 

Next, the teachers discussed whether the applet allows students to develop an appropriate 
conception of the triangle inequality theorem. 


Mary: And I guess, this question down here at the bottom does make them [students] think about 
it [triangle inequality theorem], but unless I require that they submit some kind of answer for 
that, they’re just going to ignore that. 

Bob: Mm hmm. 

Mary: And they’re going to be like, ooh smiley, not smiley. 

Bob: Yeah. There’s going to be, yeah, they’re going to see the smiley face or the no. 

Abby: Yeah. 

Mary: So we could say, if the questions at the bottom are required, and emphasized, 

Abby: then the program 

Bob: emphasis 1s placed on, on the end questions. 

Mary: Yeah. 

Abby: If the questions on the end are required and emphasized, then the tool does 

Mary: could 

Abby: could, yeah, provide 

Mary: an appropriate thought process. 


The teachers also stated that if these questions are ignored, the students will think very little 
about the theorem. 

The teachers then considered the mathematics of the applet focusing on the restrictions on the 
segments lengths and the types of triangles can be formed. Abby asked, “Does it provide all cases? 
Yes.” Mary responded, “The slider can be manipulated to any length.” Bob interjected, Well, 
between zero and ten.” Abby then asked, “Can it be an obtuse angle?” Bob responded, “Yeah, we can 
have an obtuse. Within. Yeah”. As Bob created an obtuse triangle, the teachers discussed the merits 
of the segment lengths ranging from 0 to 10, specifically the length of a segment being zero. Abby 
said, “Um, yes the slider can be moved to any length from zero to ten. Students could potentially find 
a problem with side length of zero.” She continued, “However, a discussion could be had about this 
issue.” Mary and Bob agreed. 

The teachers discussed whether the applet has a balanced difficulty level; whether all students 
would be able to use the applet to make appropriate meanings of the applet. Abby asked, “Does it 
have a balanced difficulty level?” Mary responded, “Yes, it’s not too easy.” Abby stated “Yes, not 
too easy not too hard.” 

When considering which applet they might select to use with students, the teachers compared the 
affordances and limitations of each of the four online applets. 


Bob: If we were going to use an applet, we would use, what was it, the sec, the third one, the one 
that created the triangle for you. Even though it created the [tan] triangle for you, you could 
still discover the [triangle] inequality [theorem]. 

Abby: Overall if we wanted our students to develop the inequality on their own, tool three would 
allow, um, would, um, would allow 
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Mary: for 

Abby: freedom? 

Mary: freedom in knowing when triangles can and cannot be created. And then possibly talking 
about why. 

Abby: Alright, what’d you, I’m sorry. Okay. 

Mary: Uh, knowing when triangles can and cannot be created, then a discussion could be had 
about why this is. 


When asked which applet they would use with eighth grade students, Mary, Bob, and Abby 
strongly considered selecting this applet but chose to use a different applet. 


Examples and Analysis 
In the subsequent paragraphs, we provide examples of each of the framework’s codes using Bob, 
Mary, and Abby’s evaluation of the online applet and a brief analysis of their evaluation using the 
framework. 


Attending 

When evaluating the technology, teachers attend to particular features of the technology. We 
categorized these features into five codes (see Table 1). Upon opening the website, Abby attended to 
the instruction included in the applet by reading it aloud, and Bob continued to attend to it by saying, 
“Well it’s unnecessary that it [instruction] tells you to move the line segments...” Mary, Bob, and 
Abby also attended to a given question. Abby said, “And I guess, this question down here at the 
bottom does make them [students] think about it [triangle inequality theorem]... ”” We coded these 
actions of attention as instructions and questions because the focus was on these particular features. 
At times, Mary, Bob, and Abby attended to mathematical features displayed in the applet such as 
angles and side lengths of the triangle. For example, Abby said, “In terms of the mathematics. I think 
that in this case with the angle measures...” Abby also attended to aesthetics when she noticed 
segments’ color of the triangle by stating, “Okay, yes the use of three colors...” In addition, the 
teachers would attend to interaction by dragging or clicking as they explored the abilities and 
limitations of the applet. For example, dragging segments in the applet, Bob said, “See here when 
they’re the same, it does, it shows you if you move your triangle, and then it shows point C as, well, 
so it shows that...” Attending to the interaction 1s unique to this particular framework because in 
other noticing studies, the analysis of teacher attending was mostly based on the teachers’ verbal or 
written comments (e.g., Jacobs et al., 2010; Wager, 2014). In their evaluation, the teachers did not 
attend to supportive features, which is likely due to the fact that these features were not present in the 
applet. Other applets they evaluated included these kinds of features (e.g., a video demonstration) 
and the trio did attend to those features when they were present. While evaluating this particular 
applet, other trios did take note that the applet lacked supportive features. 


Interpreting 

When teachers provide their thoughts and ideas about features of the technology and the 
technology as a whole, we refer to these actions and statements as /nterpreting. In this category, 
teachers reflect on their own uses of technology, consider how students may use the technology and 
anticipate how that affects students’ learning of key mathematical concepts. Our analysis generated 6 
codes for interpreting (see Table 1). Abby interpreted student engagement, when she said the angle 
markings could distract students from learning the theorem “because they are not essential to the 
development of the theorem.” She wanted students to be engaged in learning the triangle inequality 
theorem and thought the angle markings may distract students from staying focused on the goal of 
the activity. Often Mary, Abby, and Bob would interpret whether a feature had value. When Bob 
realized there was no reason to move the segments because the tan triangle appeared, Mary said, “It’s 
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very lame.” Mary was interpreting the value of the tan triangle or, in this case, the lack thereof. There 
were times when Mary, Abby, and Bob interpreted the mathematics of the technology including the 
display of mathematics and how students could interact with mathematical ideas. For example, Abby 
noted the lengths of the segments could range from 0 to 10 and “students could potentially find a 
problem with side length of zero.” Abby was anticipating students’ difficulty with this mathematical 
issue. In one instance, Mary and Abby interpreted whether the technology could be used for 
differentiation, when they were attempting to determine whether the technology had “a balanced 
difficulty level”. Abby also interpreted the technology’s design when she noted the use of color and 
the smiley face as features that could positively influence student learning. Finally, Mary, Abby, and 
Bob interpreted student thinking when Abby and Mary discussed whether they would select this 
applet. Abby said, “Overall if we wanted our students to develop the inequality on their own, tool 
three would allow, um, would, um, would allow...” Mary continued, “freedom in knowing when 
triangles can and cannot be created.” Abby and Mary considered how students could develop 
conjectures related to theorem based on their uses of the tool. 


Responding 

Based on their interpretation about features of the technology, teachers make possible or potential 
curricular decisions. We refer to these actions as Responding. In this phase, we defined 4 codes for 
the activities in which teachers might engage: Choose, Incorporate, Redesign, and Adapt (see Table 
1). When comparing four applets, Bob argued he would choose this applet to use with eighth grade 
students. He said, “If we were going to use an applet, we would use, what was it, the sec, the third 
one, the one that created the triangle for you. Even though it created the [tan] triangle for you, you 
could still discover the [triangle] inequality [theorem].” When playing with the applet, Mary, Abby, 
and Bob found that a smiley face appears when the segments formed a triangle. Bob said, “It would 
be better if it gave you a frowny face if it didn’t work.” We coded his idea as Redesign because Bob 
considered how he could change the tool itself. When Mary, Abby, and Bob discussed whether the 
applet helps students develop an appropriate mathematical conception of the triangle inequality 
theorem, Mary said teachers need to emphasize the bottom questions in order to facilitate students’ 
deeper understanding of the theorem. Mary’s decision to emphasize the question did not change the 
applet itself; rather she was attempting to adapt it by placing greater emphasis on the question in 
order to best meet the needs of her students and meet her goals as the teacher. Finally, when Abby 
discussed the feature that allows students to adjust the sliders such that lengths could be zero, Abby 
considered how she could incorporate the applet into her teaching. Abby said, “Students could 
potentially find a problem with side length of zero.” She continued, “However, a discussion could be 
had about this issue.” By including a discussion, Abby did not want to redesign the applet or adapt 
particular features to meet her or her students’ needs. Rather, she wanted to incorporate the applet as 
it stands, but she recognized the need to have a discussion about this particular feature. 


Analysis 

Using the framework in our analysis of Mary, Abby, and Bob’s evaluation of the applet, we 
noticed that the trio attended mostly to mathematical features, instructions and questions, and how 
they interacted with the applet. They seemed to initially focus on the features the stood out the most 
(e.g., the smiley face and the tan triangle) but progressed to less noticeable features such as the range 
of the segment lengths. The teachers’ initial interpretations were the design of the applet in which 
they focus on how the layout and features of the tool would influence students’ interaction and 
learning. Their interpretations evolved into anticipating and interpreting how students would think 
when engaging with the applet. Finally, when we coded a response as adapt or incorporate, we 
noticed that we had also coded the teachers’ interpretations as student thinking. It seems when the 
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teachers were considering how to use the applet in their own classroom, they were taking into 
account how students may think when using the technology. 


Conclusion 

We propose this framework as a device that allows researchers to examine how teachers evaluate 
and select MCTs to use in their classroom. In our evaluation of our data, we have been able to use the 
framework to document the evolution of teachers’ evaluation beginning with their initial attentions 
and ending with the selection, rejection, or modification of the tool. We believe this framework can 
potentially allow researchers to examine how the inclusion or exclusion of particular features of 
technology influences teachers’ evaluation and subsequent selection of technology. By conducting 
such an analysis with this framework, mathematics education researchers would be able to provide a 
clearer understanding of the processes teachers employ to select MCTs to use with their students. 
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